Great work @earlephilhower ! I am adopting your bearssl port in my AsyncTCP fork, and have similar observations on stack usages -- starting at 33KB of free heap space before making any connection, it quickly drains down to 2KB before a x509 decoder can be created. (the default 16KB recv buffer is the main culprit).

But I have an interesting observation during debugging:

• Because of how AsyncTCP works, data processing happens directly in the LWIP callback context, when the stack pointer is inside the "sys" context, rather than "g_cont";
• I found the sys stack usage is rather low in LWIP callbacks, only around 300 bytes
• Meanwhile, according to https://bbs.espressif.com/viewtopic.php?t=8879 the sys stack is rather large, around 8KB
• It means that, we have over 7KB of free stack space lying around unused!

Is it possible to repurpose the free sys stack to satisfy the 4.5KB heap needs of bearssl?

Very interesting idea, @Adam5Wu!

First of all, you can/should set a smaller transmit buffer size. There's no reason to allow a full 16KB transmit buffer unless you really need the highest bandwdth SSL transmit. Try the ~900 bytes transmit size I default to, it works but will break up >512b unencrypted packets into multiple 512b encrypted ones.

On receive the same logic applies. If you're not expecting, say, >2KB POSTs from a web client, then set server receive buffers smaller as well. SSL clients, though, you're kind of stuck unless you can only generate HTTP RANGE GETs..OTW 16KB receive buffers are required.

Those will save you tons of heap, but none of the stack which still needs many KB to work (not a dig on BearSSL, the encryption algorithms themselves have large temporary memory needs).

To do what you're suggesting correctly we'd need to get to the SYS context for bearssl calls, and I don't know how to do that. Do you?

To do it incorrectly (as in super-hackish) all you need to be able to do is get the end of the SYS stack. Pass that address -4.5KB in to the synthetic stack function, and it'll "just work" assuming the last 4.5KB of SYS stack aren't used. Again, I'm not sure how to grab the SYS context info as I've never thought about it...if we check before calling a BearSSL API that the space we say is available it should be fine...

Checkout libraries/ESP8266WiFi/src/include/ClientContext.h
I think the _recv() function is a callback from LWIP. Print the stack pointer:

    register uint32_t *sp asm("a1");
    Serial.printf("current stack pointer = %p\n", sp);

You should see something like 3ffffcXX, and it seems this stack can go as deep as 3fffe000.

I have tried to modified my local copy of your bearssl port, effectively undo the alternative heap changes, and it seems I am able to complete a connection under AsyncTCP, with 4.5KB more free heap. :)

I think it is possible to enjoy the benefit of extra sys stack without having to fully embrace AsyncTCP.
(Disclaimer: my understand of TLS protocol is limited) I think the most complex crypto operations happen at handshake, which also involves the heavy stack usage. So it is possible to hook into ClientContext, and ONLY intercept the hankshake part of the traffic and perform those most expensive operations in the callback context. After handshake is completed, we can continue to use the current (non-async) workflow. And because regular send/receive do not involve heavy crypto, there is no need for extra stack.

The system stack is spelled out, not explicitly, in the app.ld file. There's a line after the dram0_0_seg that's

/* __stack = 0x3ffc8000; */

So that's legit, but what we need to make it safe to use outside of the sys context in appspace is the location they stuff the stack pointer on a context switch. Since we're going to a new SDK, even, the comments on the espressif board may not hold true any longer. :(

Also, FWIW, the elliptic curve crypto takes more stack space than the x509 decode. IT's actually not the x509 portion, but the RSA 4Kbit signing/checking part of the certificate validation that eats up space early in the process. You can pull down the stack space a little bit (~1.5KB IIRC) by disabling all the elliptic curve support, but honestly I'd rather have a "works and can validate with darn near anything" than the extra 1.5KB.

In case you didn't find it already, to disable the stack stuff just go to src/inner.h, go down to the bottom of the file, and replace the STACK_PROXY_ALLOC() from ESP8266 with the non-ESP8266 one. That'll just define variables in the function like standard GCC.

ECDSA support is the main reason I pursue bearssl, so I am not going to give that up! 😄

I may be wrong, but to my understanding, RSA or ECDSA, they ONLY happen at handshake stage, when certificates are validated and shared secrets are exchanged. After that point, everything else only involves symmetric key block/stream cipher.

So my point is, during the short timespan of "after TCP connected" --> "handshake done", hook into ClientContext and perform all bearssl "heavy crypto" in the callback context, let it use the sys context stack, which can go up to 8KB. (By "use" I mean "natrual" use, not through the proxy)

Once handshake completes, the rest of bearssl operations can happen in cont context (as they do now). Because only "simple crypto" will be invoked (block/stream cipher, no RSA/ECC), the stack usage will not be high, so again, we can let it use the cont context stack natrually without proxy.

And thanks for the hint about stack proxy macros, that is just what I did. 😃

About the 2ndary stack, there's talk of using this for other things in the Arduino side as well on the maintainers gitter. We may get some official (or at least better) info on how much can safely be used.

I'll need to see how the ctx switching is handled to see if we can "easily" pop just the certificate validation in the SYS stack. The easiest way, though, would be to use the setup as-is and point that 4.5K buffer to SYS_STACKEND-4,.5KB and call it a day. Assuming that stack space is always available anyway (because the SYS stack is overallocated), that's just as safe as handling things while in SYS.

I think you're correct on the stack usage differences. Seems like only the key exchange is done using ECC or RSA, everything else is almost trivial AES or other simple block codes.

So a newbie who has been using:

#include <ESP8266WiFi.h>
#include "ESP8266HTTPClient.h"
[...]
HTTPClient http;
http.begin(url, fingerprint);

can simply switch to bearSSL exactly how...?
(currently not (knowingly) using any other libraries)

@arnolde , see the example here: https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266HTTPClient/examples/BasicHttpsClient/BasicHttpsClient.ino

hello every one I want to use setBufferSizes for BearSSL Secure Server Class which is mentioned here:
https://arduino-esp8266.readthedocs.io/en/latest/esp8266wifi/server-class.html
I am not sure how to do it and I couldn`t find any document about this application. can annyone help me about this?